White region identification to accelerate image processing for copies

ABSTRACT

The present invention solves the problem set forth above by providing a method for accelerating the production of a standalone copy of a document by a peripheral device. The invention employs a white space map of the document to be copied. The map is generated by the ASIC chip and is used by the device to identify areas of the outputted copy that do not require ink or toner. This reduces the processing resources necessary to print the copy, thus speeding up copy production.

REFERENCE TO PENDING PRIOR PATENT APPLICATION

This patent application claims benefit of prior U.S. Provisional PatentApplication Ser. No. 60/535,397, filed Jan. 9, 2004 by Derek T. Waltonfor WHITE REGION IDENTIFICATION TO ACCELERATE IMAGE PROCESSING FORCOPIES.

The above-identified patent application is hereby incorporated herein byreference.

FIELD OF THE INVENTION

This invention relates to office automation products in general, andmore particularly to printers, fax machines, scanners, copiers and thelike. Even more particularly, this invention relates to the integrationof formerly separate functions into single devices such as thosesometimes referred to as MFPs (Multi-Functional-Peripherals), and theintegration of multiple functions into a single ASIC(Application-Specific Integrated Circuit) such as those produced byOasis Semiconductor, Inc. of Waltham, Mass.

BACKGROUND OF THE INVENTION Image Processing For Copies

Performing a standalone copy on a peripheral device requiressophisticated and computationally complex processing. Typically, theinput for the copy is raw analog or digitized values from an imagesensing device. The output of the copy processing is typically either:(1) a raster ordered set of bitonal pixels, which can be delivereddirectly to a laser (as in the case of a toner based laser engine) forcreation of an image on an output media; or (2) a collection of “swaths”of bitonal pixels which is derived from the raster ordered set by aninkjet formatter, which, taken together with paper feed operations,create an image on output media (as in the case of an ink jet engine).

The processing performed to transform input to output for the copyoperation typically consists of the following steps:

(1) Black level and shading correction (compensates for non-uniformityof sensor elements);

(2) Gamma correction;

(3) Photo/text separation;

(4) Filtering;

(5) Horizontal and vertical scaling;

(6) Contrast and brightness adjust;

(7) Color space conversion (for instance, RGB→CMYK);

(8) Compensation for tonal response of writing system;

(9) Halftoning (for instance, converts 8 bit continuous tone componentsinto 1 bit bitonal components); and

(10) Ink jet formatting (reorganizes raster image data into swathssuitable for target ink jet heads, not applicable for laser based printengines).

For a particular peripheral device, particularly a multifunctionalperipheral device (“MFP”), the speed in which the above computations toconvert input to output can be performed will influence the speed withwhich the device can perform a standalone copy. Alternatively, considerthat in order to meet a particular copy speed, it is necessary toallocate resources (hardware, CPU time or otherwise) to ensure that theabove computations can be performed in the prescribed time period.

Prior Approaches

(1) Perform all Processing, for Entire Page, for each Copy

One approach to image processing for copying is to perform all of theprocessing, for the entire input and output image for each copy to beperformed. This of course requires that the full computational load beshouldered for each copy.

(2) Perform all Processing, for Entire Page, But “Scan Once, Print Many”

A simple optimization of the above brute force approach is possible forthe case that multiple copies of the same input page is desired. In thiscase, the scan operation, and a subset of the image processingoperations, need only be performed once. The partially transformed inputimage can be stored in memory. From this memory store, multiple printcopies can be achieved by repeatedly performing the remaining processingand print operations.

With this approach, it is not necessary to perform all of the imageprocessing for each output page printed. This can speed the outputprinting process and/or reduce the computation load.

A disadvantage of this approach is that it is often prohibitivelyexpensive to include enough memory in the MFP to ensure that thepartially processing page fit within memory.

Goals

Accordingly, there is a need for a method of speeding up imageprocessing in standalone copy production without increasing thecomputational load on the device. Computer chips that can achieve suchan improvement are needed.

SUMMARY OF THE INVENTION

The present invention solves the problem set forth above by providing amethod for accelerating the production of a standalone copy of adocument by a peripheral device. The invention employs a white space mapof the document to be copied. The map is generated by the ASIC chip,preferably at an early stage in the processing pipeline, and is used bythe device to identify white areas of the outputted copy that do notrequire subsequent image processing calculations and do not require inkor toner markings on the paper. Use of the white space map allows thesystem to reduce the processing resources necessary to image process andsubsequently physically print the copy, thus speeding up copyproduction.

In one preferred form of the invention, there is provided a method foraccelerating the production of a standalone copy of a document by aperipheral device comprising the steps of:

a. generating a white space map from a post-halftone raster image ofsaid document, wherein said white space map comprises rectangularregions of said image, and wherein each of said regions arecharacterized as requiring or not requiring any ink or toner; and

b. using said white space map prior to producing said copy to determineif ink or toner can be omitted in portions of said produced copy.

In one preferred form of the invention, there is provided a method forproducing a standalone copy of a document by a peripheral devicecomprising the steps of:

a. employing an image sensor to obtain pixel data from said document tobe copied;

b. determining whether each pixel is white;

c. generating a white space map from the determination made in step b,wherein said white space map comprises rectangular regions of saidimage, and wherein each of said regions are characterized as requiringor not requiring any ink or toner; and

d. utilizing said map to control the production of said copy.

In another preferred form of the invention, there is provided a computerchip capable of controlling image processing for the production of astandalone copy of a document by a peripheral device, wherein said chipcontrols:

a. the generation of a white space map from a post-halftone raster imageof said document, wherein said white space map comprises rectangularregions of said image, and wherein each of said regions arecharacterized as requiring or not requiring any ink or toner; and

b. the use of said white space map prior to producing said copy todetermine if ink or toner can be omitted in portions of said producedcopy.

In another preferred form of the invention, there is provided a computerchip capable of controlling image processing for the production of astandalone copy of a document by a peripheral device, wherein said chipcontrols:

a. an image sensor to obtain pixel data from said document to be copied;

b. the determination of whether each pixel is white;

c. the generating of a white space map based upon the determination madein b, wherein said white space map comprises rectangular regions of saidimage, and wherein each of said regions are characterized as requiringor not requiring any ink or toner; and

d. the use of said map to control the production of said copy.

In another preferred form of the invention, there is provided a methodfor accelerating the production of a standalone copy of a document by aperipheral device comprising the steps of:

a. generating a white space map from a raster image of said document,wherein said white space map comprises rectangular regions of saidimage, and wherein each of said regions are characterized as requiringor not requiring any ink or toner; and

b. using said white space map prior to producing said copy to determineif ink or toner can be omitted in portions of said produced copy.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention solves the problem set forth above by providing amethod for accelerating the production of a standalone copy of adocument by a peripheral device. The invention employs a white space mapof the document to be copied. The map is generated by the ASIC chip andis used by the device to identify areas of the outputted copy that donot require ink or toner. This reduces the processing resourcesnecessary to print the copy, thus speeding up copy production.

Omit Ink Jet Formatting for Regions Detected as White After Halftone

The ASIC can be configured to include the ability to create a “whitespace map” at the same time as the halftone processing step is performedin the standalone copy production process. This white space map is a lowresolution image generated from the post-halftone raster image. Eachlocation in the white space map corresponds to a region within thepost-halftone raster image. The size of the region is programmable. Eachlocation within the white space map indicates whether or not there isany toner/ink called for within the associated region of thepost-halftone raster image.

The white space map can be used to accelerate the ink jet formattingprocess. In the case of an ink jet-based device, the formatter, byconsulting the white space map prior to performing the work to generatea swath, can either: (1) determine that the entire swath can be omitted(no pixels on the entire swath); or (2) limit the extent of the swath(recognizing that white regions exist on the left and/or right sides ofthe swath).

This white space map based technique represents a great improvement forink jet copier systems because an ink jet engine is very quick to printwhite regions (i.e., no printing passes are required, just paper feed).In order to keep up with an ink jet printer in white regions, aformatter without white space map based acceleration would need veryhigh computational throughput. With white space map based acceleration,the formatter computing capacity can be sized for the more modestdemands of ink jet print engine speed during non-white regions.

This white space map based technique is particularly useful whencombined with the “scan once, print many” technique, as typically theink jet formatting task is performed for each of the output printedpages.

In the preferred embodiment of the invention, the generation of thewhite space map is performed at the beginning of the computationpipeline while the input pixel data is being received from the imagesensor. This allows the bulk of the image processing to be omitted fromwhite regions, which improves the copy performance and/or reduces thehardware resources.

Preferred Embodiment

In the preferred embodiment of this invention, the pixel input pipelineof the chip controlling the image processing process (which receivesinput data from sensor, performs black/shading correction, gammacorrection, and writes the resultant RGB pixels to memory) has beenextended to include logic which is capable of identifying white regions.

The logic consists of a triplet of R, G and B limit registers and atriplet of detection arrays (one for R, one for G and one for B). Thedetection arrays are organized such that each element in the arraycorresponds to a particular horizontal region of the input image. Eachelement in the detection arrays and can take on a value of 0 or 1. Avalue of 1 indicates that the region is all white, while a value of 0indicates that the region is NOT all white. When image scanning firstbegins, the detection array elements are initialized to 1 to indicatethat the regions are all white. As post gamma-corrected pixel componentsare received, they are compared with the limit registers. If a pixelcomponent is determined to be darker than its respective limit value,the associated element in the detection array is cleared to 0 toindicate that the region is NOT all white. The width and height inpixels associated with each detection array element is programmable.Firmware completes the process of generating a “white space map”equivalent by copying the detection array values to image memory atintervals corresponding to the height of the detection regions. Eachtime firmware completes copying the detection array element values toimage memory, the array elements are again initialized to 1 to indicatethat the regions are all white.

Once the white space map has been generated for a vertical region of animage, it can be used to control the downstream image processing andprinting tasks for the region in a manner which minimizes resource use.For the image processing pipeline of our example, white space mapsgenerated from post gamma-corrected pixels identify white regions of theimage which do not need to pass through steps 3 through 9 of the imageprocessing pipeline (i.e., photo/text separation, filtering, horizontaland vertical scaling, contrast and brightness adjust, color spaceconversion, compensation for tonal response, and halftoning). Thisreduction in image processing computation applies universally to alltypes of print engines (i.e., ink jet printers, laser printers, etc.)

For ink jet engines, the white space map can also be used to acceleratethe ink jet formatting process (step 10 in our example). The formatter,by consulting the white space map prior to performing the work togenerate a swath, can either: (1) determine that the entire swath can beomitted (no pixels on the entire swath); or (2) limit the extent of theswath (recognizing that white regions exist on the left and/or rightsides of the swath).

This white space map based technique represents a great improvement forink jet copier systems because an ink jet engine is very quick to printwhite regions (i.e., no printing passes are required, just paper feed).In order to keep up with an ink jet printer in white regions, aformatter without white space map based acceleration would need veryhigh computational throughput. With white space map based acceleration,the formatter computing capacity can be sized for the more modestdemands of ink jet print engine speed during non-white regions.

This white space map based technique is particularly useful whencombined with the “scan once, print many” technique, as typically theink jet formatting task is performed for each of the output printedpages.

It should be appreciated that it is also possible to position the whitemap generation logic at other positions within the image processingpipeline than the position indicated in our example. For instance, withadjustment of the values programmed into the limit registers, it ispossible to locate the white map generation prior to gamma correction.Positioning the white map generation downstream of the gamma correctionlogic is also possible, although such a positioning would sacrifice someof the performance benefit of the white maps in that fewer imagecomputations would be skipped for white areas.

It should also be appreciated that the white map method for accelerationof image processing for copies described in the invention applies ingeneral to all copy image processing pipelines, and not just to thespecific set and order of image processing operations making up theimage processing pipeline described above.

It should also be appreciated that it is not necessary for the entirewhite space map to be available prior to beginning the image processingtask. Rather, it is possible to take advantage of the white space map asportions of the white space map become available to effect processingfor the associated regions of the document; other portions of the whitespace map may subsequently be generated for other regions of thedocument. In a preferred construction, the white space map is used “onthe fly”, i.e., earlier-generated portions of the white space map areused for image processing as subsequent portions of the white space mapare being generated.

1. A method for producing a standalone copy of a document by aperipheral device, the method comprising: employing an image sensor toobtain pixel data from said document to be copied, the image sensorcomprising detection arrays, wherein each element in the detectionarrays can take on a binary value; determining whether each pixel iswhite by using limit values set forth in limit registers correspondingto the detection arrays; generating a white space map from thedetermination of whether each pixel is white, wherein said white spacemap comprises rectangular regions of said image, and wherein each ofsaid rectangular regions is characterized as requiring or not requiringany ink or toner; and utilizing said white space map to control theproduction of said copy.
 2. A method according to claim 1 wherein saidperipheral device is a multifunctional peripheral device.
 3. A methodaccording to claim 2 wherein said multifunctional peripheral device isan ink jet device.
 4. A method according to claim 3 wherein the imagesensor comprises a red detection array, a green detection array and ablue detection array; and wherein each element in each array detects ahorizontal region of the document, wherein a size of said horizontalregion is programmable.
 5. A method according to claim 4 wherein thepixel determination comprises determining that a pixel is not darkerthan a limit value set forth in corresponding red, green and blue limitregisters.
 6. A method according to claim 5 wherein said white space mapis generated by firmware by copying the detection array values to imagememory at intervals corresponding to the height of said horizontaldetection regions.
 7. A computer device capable of controlling imageprocessing for the production of a standalone copy of a document by aperipheral device, wherein said computer device controls: an imagesensor to obtain pixel data from said document to be copied, the imagesensor comprising detection arrays, wherein each element in thedetection arrays can take on a binary value; the determination ofwhether each pixel is white by using limit values set forth in limitregisters corresponding to the detection arrays; the generating of awhite space map based upon the determination of whether each pixel iswhite, wherein said white space map comprises rectangular regions ofsaid image, and wherein each of said rectangular regions ischaracterized as requiring or not requiring any ink or toner; and theuse of said white space map to control the production of said copy.
 8. Acomputer device according to claim 7 wherein said image sensor comprisesa red detection array, a green detection array and a blue detectionarray; and wherein each element in each array detects a horizontalregion of the document, wherein a size of said horizontal region isprogrammable.
 9. A computer device according to claim 8 wherein saiddevice comprises logic that compares a pixel value obtained from saidred, green and blue detection arrays with a corresponding red, green orblue limit register to determine if said pixel is white.
 10. A computerdevice according to claim 9 wherein said white space map is generated byfirmware by copying the detection array values to image memory atintervals corresponding to the height of said horizontal detectionregions.
 11. A method for accelerating the production of a standalonecopy of a document by a peripheral device, the method comprising:generating a white space map from a raster image of said document byemploying an image sensor to obtain pixel data from said document to becopied, the image sensor comprising detection arrays, wherein eachelement in the detection arrays can take on a binary value, anddetermining whether each pixel is white by using limit values set forthin limit registers corresponding to the detection arrays, wherein saidwhite space map comprises rectangular regions of said raster image, andwherein each of said rectangular regions is characterized as requiringor not requiring any ink or toner; and using said white space map priorto producing said copy to determine if ink or toner can be omitted inportions of said produced copy.
 12. A method according to claim 11wherein said white space map is generated prior to halftoning.
 13. Amethod according to claim 11 wherein said white space map is generatedafter black level and shading correction and before halftoning.
 14. Amethod according to claim 11 wherein said white space map is generatedafter gamma correction and before halftoning.
 15. A method according toclaim 11 wherein the white space map and image processing occur intandem, and further wherein latter portions of the white space map arebeing generated even as image processing is being effected usingpreviously generated portions of the white space map.